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Introduction 


A broad range of activities are essential to successfully implement an 
Evidence-Based Practice (EBP). These activities help build support for the EBP, 
integrate the EBP into agency policies and procedures, train staff agency-wide 
on basic EBP principles, and allow for ongoing monitoring and evaluation of the 
program. Practitioner training in how to deliver an EBP is only one aspect of 
EBP implementation. 

This booklet introduces the general range of activities involved in successfully 
implementing EBPs. For more information about implementing specific EBPs, 
see Evidence-Based and Promising Practices in this KIT, which contains 
detailed, EBP-specific information on such activities as staffing, training, 
financing, and fidelity measuring. 
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Build Support for Change: Five 
Steps for Consensus Building 


step 1 


Identify key stakeholders 
who will be affected by 
implementing the EBP. 


Within a system, change affects different 
stakeholders differently. When changing the 
mental health system, mental health agencies 
shonld expect varied reactions from staff, 
commnnity members, providers, and families 
and yonth. 

Since misnnderstanding EBPs can stand in the 
way of yonr efforts to implement EBPs, proactively 
bnilding a consensns to change the system or 
implement EBPs in the commnnity is an essential 
component of snccess. This can be done throngh 
a five-step process (see Fignre 1). 


Figure 1: Five Steps 
for Consensus Building 


To start implementing your EBP: 

1. Identify key stakeholders who will be 
affected by implementing the EBP. 

2. Invite potential champions from each 
stakeholder group to participate in an EBP 
advisory committee. 

3. Ask the committee to advise you during 
the process. 

4. Build an action plan. 

5. Actively involve committee members in 
such ongoing activities as the following: 

■ Participating in EBP basic training; 

■ Providing basic information about the EBP 
to their stakeholder groups; 

■ Advising you during all phases 

of the implementation process; and 

■ Participating in an ongoing evaluation 
of the EBP. 


Stakeholders may include provider 
agency personnel at all levels, 
mental health authority staff, family 
organizations, family members, youth, 
researchers, policymakers, and funders. 
Consensus building should also involve 
a broad array of community agencies 
in education, child welfare, juvenile 
justice, and health care. 


Step 2 


Invite potential champions 
from each stakeholder group 
to participate in an EBP 
advisory committee. 


According to agencies who have 
successfully implemented EBPs, 
identifying ongoing champions and 
forming an advisory committee are 
critical activities for success. Although 
you may feel that creating an advisory 
committee slows the process, it soon 
becomes apparent that any amount of 
time used to build stakeholder support 
is worth the effort. 


EBPs have little hope for success if 
the community fails to recognize that 
they are needed, affordable, worth the 
effort, and congruent with community 
values and the agency’s practice 
philosophy. Mental health authorities 
and agency administrators must convey 
to key stakeholders a clear vision and a 
commitment to implementing the EBP. 

By forming an advisory committee 
of potential champions from each 
stakeholder group, you will be able 
to disseminate information broadly 
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within the community. You should 
plan to build committee members’ 
knowledge in the EBP and then ask 
them to hold informational meetings 
or to disseminate information regularly 
to their stakeholder groups. 


Step 3 


Ask the committee to advise 
you during the process. 


In addition to having them conduct 
community education activities, ask 
committee members to advise you 
during all phases of the implementation 
process. This is why the advisory 
committee should be instituted 
and members educated about the 
EBP early in the planning process. 
Community members may help assess 
how ready the community and the 
agency are to implement the EBP and 
its activities. Once the EBP is in place, 
committee members can keep EBP 
staff informed of relevant community 
trends that may have an impact on 
providing the evidence-based services. 


Step 4 


Build an action plan. 


Once key stakeholders basically 
understand the EBP, have your advisory 
committee develop an action plan for 
implementation. Action plans outline 
activities and strategies involved in 
developing the EBP program, including 
the following: 

■ Integrating the EBP principles into 
mental health authority and agency 
policies and procedures; 

■ Outlining initial and ongoing 
training plans for internal and 
external stakeholders; 

■ Designing procedures to monitor 
and evaluate the EBP regularly; and 

■ Base the activities in your action 
plan on the needs of the population 
you serve, your community, and 
your organization. 


Step 5 


Actively involve committee 
members in ongoing activities. 


EBP advisory committees are crucial 
for sustaining the EBP over time. 
When EBP staff turn over, or other 
well-trained staff leave and must be 
replaced, or when funding streams 
or program requirements change, 
community alliances are essential to 
maintain the EBP. A well-established 
committee can champion the EBP 
through changes. 


Committee members can help you with 
such tasks as deciding which outcomes 
you should emphasize. They can help 
you evaluate and integrate continuous 
quality improvements and engage in 
other essential activities. 
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Integrate the EBP into Policies, 
Procedures, and Financing 


Examine policies and procedures 

Mental health authorities and agencies that have 
successfully implemented EBPs highlight the 
importance of integrating the EBP into pohcies 
and procedures. For example, you will immediately 
face decisions about staffing the EBP program. 

Mental health authorities can support the 
implementation process by integrating staffing 
criteria into regulations. Agency administrators 
should select an EBP program leader and 
practitioners based on mental health authority 
regulations and qualifications that the EBP requires. 
New EBP position descriptions should be integrated 
into the agency’s human resource policies. 


Agency administrators and mental health 
authorities should also review administrative 
policies and procedures to ensure that they are 
compatible with EBP principles. For example, 
you may need to modify admission and discharge 
assessment, treatment planning, or service-delivery 
procedures. Make sure policies and procedures 
include information about how to identify children 
and adolescents who are most likely to benefit 
from the EBP and how to integrate inclusion 
and exclusion criteria into referral mechanisms. 

Examine policies and procedures early in the 
process. Integrating EBP principles into policies 
and procedures will build the foundation of the 
EBP program and will ensure that the program 
is sustainable. While most changes will occur in 
the planning stages, monitoring and evaluating 
the program regularly will allow you to periodically 
assess the need for more changes. 

Assessment instruments can help identify 
strengths and areas of infrastructure that may 
need reinforcement to support implementing 
and disseminating EBPs. Examples of assessment 
instruments that can be used to help integrate 
EBPs into policies and procedures are presented 
in Figure 2. 
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Figure 2: Assessments for Integrating EBPs 
into Service Systems 


The State Health Authority Yardstick (SHAY) (Finnerty, Rapp, Lynde, & Goldman, 2005) was 
developed by the New York State Department of Mental Health to assess state infrastructure to support 
EBPs. It was developed in collaboration with the SAMHSA-CMHS-funded National EBP Implementation 
Project. Corresponding directly to the infrastructure domains, the areas measured on the SHAY include: 

■ Planning for EBP implementation 

■ Financing (adequacy, startup, conversion) 

■ Training (ongoing consultation and technical support, quality, infrastructure/sustainability) 

■ Leadership (state commissioner and EBP leader) 

■ Policy and regulations of the State Mental Health Authority (SMHA) and Non-SMHA agencies 
related to EBP program standards 

■ Quality improvement (client outcomes, stakeholder support) 

The Organizational Readiness for Change (Lehman, Greener, & Simpson, 2002) instrument was 
developed for use in substance abuse treatment organizations to assess the readiness of an organization 
to implement EBPs. This tool also affords opportunities for use in children's mental health service 
provider organizations. There are two versions—one for directors and another for staff—which assess 
18 domains in the following areas: 

■ Motivational readiness (perceived need for improvement, training needs, pressure for change) 

■ Institutional resources (office, staffing, training, resources, computer access, electronic communication) 

■ Staff attributes (value placed on professional growth, efficacy, willingness and ability to influence 
co-workers, adaptability) 

■ Organizational climate (clarity of mission and goals, staff cohesiveness, staff autonomy, openness 
of communication, level of stress, openness to change) 

Measures are available at http://www.ibr.tcu.edU/pubs/datacoll/commtrt.html#Form-ORC. 

The Evidence-based Practice Attitude Scale (Aarons, 2004) was designed to assess mental health 
provider attitudes toward adopting evidence-based practices. This brief measure assesses four dimensions 
related to the following: 

■ Appeal of evidence-based practices 

■ Likelihood of adopting given requirements to do so 

■ Openness to new practices 

■ Perceived divergence of EBPs from usual practices 
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Identify funding issues 

Identifying and addressing financial barriers is 
critical because specific costs are associated with 
starting new EBP programs and sustaining them. 
Identify short- and long-tenn funding mechanisms 
for EBP services, including federal, state. Private 
foundation funds are also important. You can use 
your EBP advisory committee to project startup 
costs by identifying the following: 

■ Time for meeting with stakeholders that 
is not reimbursed; 

■ Time for staff while in training; 

■ Costs associated with reducing productivity 
requirements to account for time spent 
planning; 

■ Travel to visit other model EBP programs; and 

■ Costs for needed technology (cell phones and 
computers) or other one-time expenses accrued 
during the initial implementation effort. 

You should also identify funding mechanisms for 
ongoing EBP services and tlie support of continuous 
quality improvement efforts, including ongoing 
training, supervision, technical assistance, fidelity, 
and outcomes monitoring. In addition, you may 
need to revise rules for reimbursement that are 
driven by service definitions and criteria. This may 
require interagency meetings at the federal, state, 
and local levels. Besourees that will help you plan 
for financing issues are presented in Figure 3. 


Figure 3: Resources for Exploring 
Financing Options 


■ A Self Assessment and Planning Guide: 
Developing a Comprehensive Financing 
Plan (Armstrong et al., 2006). 

This guide was developed to increase 
understanding of financing structures 
and strategies to support effective systems 
of care. It was designed to guide service 
systems and individual communities in 
assessing their current financing structures 
and strategies and to prioritize a strategic 
financing plan for moving forward. It 
provides a means for projecting possible 
outcomes that are to be achieved and 
strategies for achieving those outcomes. 

■ Public Financing of Flome and Community 
Services for Children and Youth with 
Serious Emotional Disturbance: Selected 
State Strategies (Ireys et al., 2006). 

This monograph, sponsored by the 
U.S. Department of Flealth and Human 
Services, provides information about 
sources of federal funding for child 
mental health services and profiles state 
approaches to financing home and 
community based services, including 
various Medicaid options. 
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Understand Medicaid 

Medicaid is an essential resource for funding many 
EBP programs, and understanding how it works is 
essential for agencies planning to adopt an EBP. 
Medicaid is a federal-state partnership. The federal 
government develops regulations based on federal 
statutory requirements. State Medicaid agencies 
have the responsibility to implement the program 
in their state. 

The federal statute holds the Medicaid state 
agency and the Medicaid director responsible for 
all aspects of the program. While Medicaid state 
agencies may collaborate and contract with other 
public and private agencies, including mental 
health agencies, the Medicaid director is held 
accountable for all activities. 

Medicaid now funds more than half of public 
mental health services administered by states 
and could account for two-thirds of such spending 
by 2017 (Buck, 2003). Accordingly, Medicaid 
is the single largest source of funding for public 
mental health services for children, youth, and 
their families. 


Medicaid eligibility 

Medicaid provides access to health coverage for 
low-income women and their children and for 
people with disabilities and others who have high 
medical costs. To qualify all these individuals 
must have low or moderate incomes, but Medicaid 
eligibility is also linked to age, with specific eligibility 
categories for elderly people as well as children. 

Since Medicaid is a means-tested program, it 
has extensive rules on income and resources. 

In addition, individuals must fit into one of the 
eligibility categories established by federal law and 
meet other criteria, such as residency requirements 
and citizenship or immigration status. Children 
are the largest age group covered under Medicaid 
and represent about half of all beneficiaries. 


Medicaid-covered services 

The Medicaid benefits or services package is 
broad. More than 30 services listed in Medicaid 
Statute Section 1902(a) are either optional or 
mandatory. If they are mandatory, states must 
provide these services. In addition, states may 
choose to cover certain optional services. 

In both cases. Federal Financial Participation 
(FFP) is available as a match from the federal 
government to the states for services provided. 

A number of broad-based services that are covered 
are extremely important for maintaining an 
adequate mental health care system and that in 
fact are the basis for the entire health care system. 
In the following material, Medicaid services that 
are important for mental health service delivery 
are discussed and defined. 
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Physician services 

These are services provided by psychiatrists; 
primary care physicians such as pediatricians, 
family practitioners, and internal medicine 
physicians; and speciahsts, including those in 
neurology, obstetrics and gynecology (OB-GYN), 
and surgery. 

Services that are important for children and youth 
can include screening for mental health issues and 
physical examinations to rule out physical health 
problems that can mimic mental health issues, 
such as hyperthyroidism and Attention Deficit 
Hyperactivity Disorder (ADHD).^ Neurological 
issues such as temporal lobe seizures can mimic 
mental health issues and should be considered. 
Payment for these services is covered under 
Medicaid for eligible individuals. 

Psychiatry services for therapy and medication 
monitoring are common reasons for providing 
mental health services. Medicaid expenditures 
for physician services are second only to those for 
pharmaceuticals for all age groups. See Medicaid 
statute 1905(a)(5)(A) and (B) and regulation 
42CFB440.50. 


Inpatient hospital services 

These services include general hospitals and 
specialty hospitals such as mental health and 
children’s acute care hospitals. The acute care 
hospitals can provide a full range of services: medical, 
surgical, obstetrics, and inpatient mental health; 
but not for Institutions for Mental Disease (IMD). 

For children and youtli, tliis inpatient care can also 
include long-term mental health hospitalization. 
See Medicaid statute 1905(a)(1) and regulation 
42CFB440.20. 


' Some states consider ADHD as a mental health condition. 


Federally Qualified Health Centers (FQHC) 

These facilities provide comprehensive ambulatory, 
community-based services in medically underserved 
areas. Services include adult medicine, pediatries, 
OB-GYN, pharmacy, laboratory and radiology, and 
mental health services by referral. 

Gase management is provided. A number of 
FQHGs provide co-located physical and mental 
health services. An example is the FQHG in 
Greenwood, North Garolina. Federal Medicaid 
payment regulations require enhanced 
reimbursement rates to cover the cost of providing 
comprehensive health and, where applicable, 
mental health services. See Medicaid statute 
1905(a)(2)(G) and 1905(L)(1) and (2) and 
regulation 42GFB491.l-491.il. 


Rural health clinic 

These facilities provide health services in rural and 
medically underserved areas. Services may or may 
not be as comprehensive as those offered through 
FQHGs. See Medicaid statute 1905(L)(1) and (2) 
and 1905(I)(1) and regulations 42GFB440.20(b) 
and (c). 


Laboratory and X-ray services 

These are often used as part of a comprehensive 
assessment of children and youth. Laboratory 
services may be used to ensure that the use of 
psychotropic medications is safe and that the 
liver, blood, and kidney functions are adequate 
to metabolize these medications. They can also 
be used to rule out physical health diseases that 
can mimic mental health issues (for example, 
hyperthyroidism). See Medicaid statute 1905 (a) 
(3). Various mechanisms for financing services 
through Medicaid are presented in Figure 4. 
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Figure 4: Range of Medicaid Financing Mechanisms for Home and Community-Based 
Services for Children and Youth 


Clinic Option 

States can provide non-hospital-based community 
services, but only in community clinics and under 
the direction of a medical doctor. 

■ Services typically include traditional counseling, 
psychotherapy, and medication management. 

■ Not typically used to expand home and community- 
based services because of the limitation in setting. 

Early Periodic Screening, Diagnosis, and Treatment 
(EPSDT) 

■ As part of regular screening for physical and mental 
health conditions of Medicaid eligible children under 
21 years of age, states are required to provide services 
that are federally authorized by Medicaid, but not 
necessarily covered in a State Medicaid Plan. 

■ States have not been using proactively to expand 
home and community-based services. 

Managed Care Waivers 1915 (b) and Demonstration 
Waivers 1115 

■ Managed care mechanism "carved" out from 

the regular state plan that limits choice of provider 
under Medicaid. 

■ Managed care entity can use funding streams 
beyond Medicaid. 

■ 1915 waivers are approved for 2 years and states 
can renew. 

■ Demonstration waivers (1115) allow states to innovate 
through expanding eligibility or services, not typically 
covered by Medicaid, or to test innovative service 
delivery systems. 

Medicaid 1915 (c) Home and Community-Based 
Services Waivers 

■ States can expand Medicaid coverage of community- 
based services, not otherwise covered, for a designated 
number of individuals as an alternative to institutional 
care (that is, hospital). 

■ States can expand Medicaid eligibility to 
populations, not otherwise eligible for Medicaid 
(for example, uninsured). 


Rehabilitation Option 

■ Rehabilitation service defined as "any medical 

or remedial services (provided in a facility, home, 
or other setting) recommended by a physician 
or other licensed practitioner...for the maximum 
reduction of physical or mental disability and 
restoration of an individual to the best possible 
functional level." 

■ Services can include: 

• Restoration and maintenance of daily living skills; 

• Training in social skills; 

• Development of appropriate social networks; 

• Recreational services that are therapeutic in nature; 
and 

•Telephone counseling services. 

■ Wide variation in services provided by states under 
the Rehabilitation Option. Some states cover: 

• Psychological assessment; 

• Crisis intervention; 

• Individual, group, family therapy; 

• Day treatment; 

• Home-based services; 

• Behavioral management skills training; 

•Therapeutic foster care; 

• Family preservations services; 

•Care coordination; and 

• Help in medication compliance. 

TEFRA Katie Beckett Provision 

States can expand Medicaid eligibility to children with 
disabilities by waiving parental income for children who 
are living at home but who would otherwise be eligible 
for Medicaid-funded institutional care. 


(Summarized from Ireys, Pires, & Lee, 2006) 
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Figure 4a: What Do State Medicaid Plans Cover? 


■ Varies from state to state. 

■ See Medicaid Plan for your state. 

■ For summary of clinic services covered by Medicaid in your state in 2004, see: 

http://www.kff.org/medicaid/benefits/service.jsp7gr = off&nt = on&so = O&tg = O&yr = 2&cat = 12&sv = 5. 

Guidance 

■ Include State Medicaid agency, other healthcare funders, and managed care organizations 
in discussions early. 

■ Read more about the various mechanisms described above. 

■ In preparing to work with State Medicaid agency: 

• Clearly define the EBP being considered. 

• Identify components of the EBP (match what is already covered). 

• Describe qualifications of practitioners. 

• Describe the dose and duration of the service. 

• Present cost-effectiveness data, if available (See Washington State Institute for Public Policy 
report on Benefits and Costs of Prevention and Early Intervention Programs for Youth at: 

http://www.wsipp.wa.gov/pub.asp7Docid = 04-07-3901). 

References 

Ireys, H.T., Pires, S., & Lee, M. (2006). Public financing of home and community services for children 
and youth with serious emotional disturbances: Selected state strategies. (2006). Washington, DC: 
Office of Disability, Aging and Long Term Care Policy, Office of the Assistant Secretary for Planning 
and Evaluation, U.S. Department of Health and Human Services. Center for Medicare and Medicaid 
Services.(2005). Medicaid support of evidence-based practices in mental health programs. Available at: 
http://www.cms.hhs.gov/PromisingPractices/Downloads/EBP_Basics.pdf. 
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State and program examples 
of Medicaid funding 

States that are actively trying to expand coverage 
for intensive home and commnnity-based services 
typically nse a combination of Medicaid financing 
options in addition to other mechanisms, inclnding 
contracting with care management entities for 
high-risk popnlations and nsing blended or braided 
funds from other agencies, including Medicaid 
(Ireys et ah, 2006). See Figure 5 for an illustration 
of Michigan’s Home- and Community-Based 
Waiver Program. 

Other examples of state programs include the 
following: 

■ Five states with home and community-based 
waivers (IN, KS, NY, VT, and WI) cover parent/ 
family/home education, support, and training 
(Ireys et ah, 2006). 

■ Two states with home- and community-based 
waivers (VT & WI) cover therapeutic foster care 
(Ireys et ah, 2006). 


Multisystemic Therapy Services, the disseminators 
of MST, developed a position statement on 
Medicaid funding for MST that outhnes 
strengths and weaknesses of Medicaid as a 
funding source. To find the document, go to: 
http://www.mstservices.com/userauth/Medicaid%20 
Standards.htm. 

The website also contains information on each 
state’s position on funding MST with Medicaid 
dollars. For states that do fund MST with Medicaid 
(for example, Arizona, California, Indiana), 
between 40 and 60 percent of a model program 
budget is covered. 
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Figure 5: Illustration of Michigan's 1915[c] Home and Community-Based Waiver 


The Home and Community-Based waiver, (1915[c] waiver), for children with serious emotional 
disturbance (SED) is administered by the Michigan Department of Community Health (MDCH) and 
funded with federal Medicaid dollars matched by local resources including the state general funds 
allocated to the Michigan Community Mental Health Services (CMHSP) program. 

The waiver is designed to provide in-home services and supports to children under age 18 with SED 
who meet the criteria for admission to a state inpatient psychiatric hospital and who are at risk of 
hospitalization if waiver services are not provided. The waiver is limited to children residing in counties 
that that have been pre-approved through the waiver. Under Medicaid statutory and regulatory 
requirements, all matching funds must be local funds. 

Examples of local funds used in this waiver are CMHSP general revenue funds, local child care funds. 
However, Title IV-E Foster Care (SSA 45CFR Parts 1355, 1356, 1357) funds cannot be used as match 
as they are federal funds. To ensure that all matching funds are local, CMHSP must document the type 
and source of funds used to meet the match obligation. The documentation is provided in both the 
individual child's budget and in the written agreement between the State agencies: CMHSP and MDCH. 

Eligibility criteria for the program include the following. The child must: 

■ Be at risk of hospitalization in the state psychiatric facility, 

■ Demonstrate serious functional limitations. The criteria will be identified using the Child 
and Adolescent Functional Assessment Scale (CAFAS®), 

■ Be under 18 years old, 

■ Be financially eligible for Medicaid when viewed as a family of one (that is, Katie Beckett option) 
or otherwise eligible for Medicaid, and 

■ Be in need of and receive at least one waiver service per month. 

Birth and adoptive families must choose these services as an alternative to hospitalization, participate 
in developing the plan of service, allow services to be provided in the home setting, and provide care 
and supervision beyond the services authorized in the waiver. 

The services that are provided use the "Wraparound" as a framework for providing and coordinating 
services that are family-centered. It will also include the services that are traditionally provided such 
as physician services, medication management, family therapy, i.e., functional family therapy. 

Functional family therapy can be provided in the community and in the home. If the family members 
are Medicaid eligible then the services can be provided through Medicaid. 

Reference 

Michigan Department of Community Health, SED Waiver, A Home and Community-Based Waiver for 
Chiidren with Serious Emotionai Disturbance, A Technicai Assistance Manual, May 2007, taken from 

http://michigan.gov /documents/mdch/SED_Waiver_TA_Manual_5-9-07_FlNAL_196150_7.pdf). 
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Train All Levels of Staff 
in the Agency 

One of the next steps in implementing yonr 
EBP is to develop a training plan. Yon may gange 
the amonnt of training needed by assessing the 
readiness of yonr commnnity. If a commnnity is 
nninformed abont the EBP and is nnaware of tlie 
existing need, yon may have to condnct a wide range 
of edncational activities. If a commnnity already 
nnderstands the EBP and knows how it may address 
problems that commnnity members want to solve, 
you may need fewer educational activities. 

Agency administrators who have successfully 
implemented EBPs highlight the importance 
of providing basic training on the EBP to all levels 
of staff throughout the agency. Educating and 
engaging staff will ensure support for the EBP. 

In the long run, if they are well trained, EBP 
staff will have an easier time obtaining referrals, 
collaborating with staff from other service 
programs, and facilitating a continuum of care. 

You can help train key stakeholder groups if you first 
train members of your EBP advisory committee and 
then ask them to disseminate information about the 
purpose and benefits of the EBP. 

Ongoing inservice training is an efficient way to 
provide background information, the EBP practice 
philosophy and values, and the basic rationale for 
EBP service components in a comfortable training 
environment. Consider including members of your 
advisory committee in decisions about the frequency 
and content of basic EBP training. 


Offer more intensive training to 
program leaders and practitioners 

While staff at all levels in the agency should receive 
basic EBP training, the EBP program leader and 
practitioners will require more intensive training. 
To help practitioners integrate EBP principles into 
their daily practice, offer comprehensive skills 
training to those who provide EBP services. 

For information about training requirements 
and resources for the EBPs covered in this KIT, 
see Evidence-Based and Promising Practices. 

Although most skills that practitioners need may 
be introduced through formal training, research 
and experience show that the most effective way 
to teach EBP skills is through supervision, on-the- 
job consultation, and coaching (Fixsen et ak, 2005). 

In many mental health agencies, turnover is high. 
This means that a single training will not be 
sustained unless the new expectations are 
incorporated into ongoing training efforts for 
new employees. Early in the process, mental health 
authorities and agency administrators must decide 
how to do the following: 

■ Identify internal and external stakeholders 
who will receive basic training; 

■ Determine how often basic training will 
be offered; 

■ Identify who will provide the training; 

■ Identify EBP staff and advisory group members 
who will receive comprehensive skills training; 

■ Determine the training format for ongoing 
training to EBP staff; 

■ Determine whether EBP staff may visit a model 
EBP program; and 

■ Determine how consultation and coaching 
will be provided. 


Many agencies have found it useful for EBP 
program leaders and practitioners to become 
famihar with the structure and processes of the 
practice by visiting agencies that have successfully 
implemented the EBP. 
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Monitor and Evaluate Regularly 

Key stakeholders who implement EBPs may find 
themselves asking two questions: 

■ Has the EBP been implemented as planned? 

■ Has the EBP resulted in the expected 
outcomes? 

Asking these two questions and using the answers 
to improve your EBP program are critical 
components for ensuring the success of your 
EBP program. 

■ To answer the first question, collect process 
measures, which capture how services are 
provided. 

■ To answer the second question, collect outcome 
measures, which capture the program s results. 

As you prepare to implement an EBP, it is strongly 
recommended that you develop a quality assurance 
system using both process and outcome measures 
to monitor and improve the program’s quality from 
the start through its mature development. 


Why you should collect process measures 

Process measures give agency staff an objective, 
structured way to gain feedback about program 
development and about how services are provided. 
Experience suggests that this is an excellent 
method to diagnose program weaknesses and 
to clarify strengths. 

Process measures also give mental health 
authorities a comparative framework in which 
to evaluate statewide implementation of an EBP. 
They allow mental health authorities to identify 
statewide trends and outliers. Once EBP programs 
reach high fidelity, ongoing monitoring allows 
agency staff to test local innovations while ensuring 
that EBP programs do not drift from core EBP 
principles. (See Figure 6.) 
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Figure 6: What Does Fidelity Mean? 


Sometimes an evidence-based practice does not produce expected outcomes because it is not being 
implemented according to the model (the model is the version of the intervention that research found 
to be effective). A practice is not implemented well or "according to the model" when critical features 
or components of the intervention are not included in the version of the EBP being implemented. 
Research has shown the EBP to be effective if key components of the intervention are in place. 

When these components are absent, the version of the EBP being implemented is no longer "true 
to the model." It lacks fidelity. 

Fidelity refers to the degree that the version of the EBP being implemented is "true to the model." 

If most of the key components are present, the implemented version has high fidelity; if most are 
absent, it has low fidelity. 

Fidelity scales are used to measure the degree to which the critical components of an EBP are present. 
The reason such scales are important is that they are like a thermometer—they tell you if the EBP is 
being implemented as it should be or if adjustments need to be made. Often, if an intervention is not 
producing desired outcomes, a clinician will recommend an alternative. Before switching interventions, 
it helps to make sure that the intervention was properly designed and administered. Fidelity scales 
indicate any modifications that are needed. In this sense, fidelity measures are a gauge of the quality 
of services that consumers and family members receive. 

For many interventions, fidelity scales do not exist. In this situation, standards and clinical guidelines 
are used to assure quality instead of fidelity measures. 
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Why you should collect outcome measures 

While process measures capture how services are 
provided, outcome measures capture the program’s 
results. Every mental health service intervention — 
whether considered treatment or rehabilitation — 
has both immediate and long-term client goals. 

In addition, children and families have goals for 
themselves, which they hope to attain by receiving 
mental health services. These goals translate 
into outcomes and the outcomes translate into 
specific measures. 

Some outcomes result directly from an intervention, 
such as staying in school. Others are indirect, such 
as improving a family’s quality of life as a result 
of the child being able to stay in school. Some 
outcomes are concrete and observable, such as 
the number of days attending school in a month. 
Others are subjective and private, such as being 
satisfied with EBP services. 


Family and child outcomes are the bottom-hne 
for mental health services, which is analogous 
to the role of profit in business. No successful 
businessperson would assume that the business 
was profitable just because the enterprise 
produced a number of widgets or because 
employees worked hard. Productivity does 
not necessarily lead to profit. 


Assessing child and family outcomes 

A review of the most frequently identified client 
outcomes assessed across the EBPs contained in 
this KIT included reduction of disruptive behavior, 
improved family functioning, the reduction of 
delinquent behavior, and improved parenting skills 
(Zubritsky et ah, 2007). Table 1 summarizes the 
instruments for measuring these types of outcomes 
and information about how these materials can 
be acquired. 
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Table 1: Measurement Instruments for Assessing Child and Family Outcomes 


Instrument 

Age 

Range 

Outcomes Assessed 

#of 

Items 

Informant 

Reference/Publisher 

Child Behavioral Symptoms 

Behavioral Assessment System 

2-21 

Aggression, conduct 

100-160 

Parent, teacher, child 

2nd Ed: Reynolds & 

for Children (BASC-2) 


problems 


(8-21) 

Kamphaus, 2004* 
http://ags.pearsonassessments. 






com 

Achenbach System of Empirically 

1.5-18 

Aggressive behavior 

99-118 

Parent, teacher, child 

Achenbach & Rescorla, 2000, 

Based Assessment 




(11-18) 

2001* http://www.aseba.org 
http://www.assess.nelson.com 

Early Childhood Inventory-4 & 

3-18 

Oppositional behavior. 

108 

Parent, teacher, child 

Sprafkin & Gadow, 1996* 

Children's Symptom lnventory-4 


oppositional defiant disorder, 


(12-18) 

Gadow & Sprafkin, 1995* 



conduct disorder 



http://www.checkmateplus.com 

Conners' Rating Scales: 

3-17 

Oppositional behavior 

27-80 

Parent, teacher, child 

Conners, 1997* 

Long & Short Form 




(12-18) 

http://www.pearsonassessments. 






com 

Revised Behavior Problem 

Checklist 

5-18 

Conduct disorder 

89 

Parent & teacher 

Quay & Peterson, 1996* 

Eyberg Child Behavior Inventory 

2-16 

Conduct disorder. 

36 

Parent 

Eyberg, 1999** 

(ECBl) 


oppositional defiant disorder 




Strengths and Difficulties 

3-16 

Strengths 6- difficulties 

25 

Parent, teacher, child 

Goodman, 1997 

Questionnaire (SDQ) 




(11-16) 

Goodman, Meltzer, Bailey, 1989 

http://www.sdqinfo.org 

http://www.routledge.com 

Peabody Treatment Progress 

11-18 

Global measure of severity 

33 

Youth, adult, caregiver. 

Bickman et al., 2007 

Battery: Symptoms & Functioning 


of symptoms of conduct 


and clinician 

http://peabody.vanderbilt.edu/ 

Severity Scale 


disorder and oppositional 
defiant disorder 



ptpb 

Child Functioning 

Child and Adolescent Needs 

4-21 

Oppositional behavior. 

41 

Clinicians 

Lyons, 1999** 

and Strengths- Mental Health 


antisocial behavior 



http://www.buddinpraed.org/ 

(CANS-MH) 





cans/ 

The Child and Adolescent 

5-18 

Aggression, conduct disorder, 

165 

Clinicians 

Hodges, 2000, 2004 

Functioning Scale (CAFAS) 


oppositional defiant disorder, 
behavioral non-compliance 



http://www.CAFAS.com 

Parent/Family Assessment 

Dyadic Parent-Child Interaction 

3-6 

Quality of parent-child social 


Structured observation 

Eyberg, Nelson, 

Coding System III 


interactions 


of parent-child 

Duke, Boggs, 2005 





interaction 

http://pcit.phhp.ufl.edu/ 

Measures.htm 

Behavioral Coding System 


Quality of parent-child 


Structured observation 

Forehand & McMahon, 1981 



interactions 


of parent-child 

McMahon &■ Forehand, 2003, 





interaction 

(p. 58). 

Caregiver Wish List 

3-18 

Caregiver's parenting skills 

67 

Caregiver, parent 

Hodges, 2002 



and child's behavioral 
compliance 



http://www.CAFAS.com 

Parenting Scale 

1.5-4 

Parenting skills 

30 

Parent 

Arnold, O'Leary, Wolff, & 

Acker, 1993 


*ln McMahon, R., & Frick, P. (2005). Evidence-based assessment of conduct problems. Journa/ of Clinical Child and Adolescent Psychology, 34 (3), 477-505. 

* * In Grisso, T., &■ Underwood, L. (2003). Screening and assessing mental health and substance use disorders among youth in the juvenile justice system: Research 
and Program Brief. Delmar, NY: National Center for Mental Health and Juvenile Justice. 
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Additional sources for measurement instruments 
include the following; 

Maruish, M. (Ed.). (2004). The Use of 
Psychological Testing for Treatment Planning 
and Outcome Assessments: Instruments for 
Children and Adolescents (3rd ed.). Mahwah, 
NK: Lawrence Erlbaum Associates, Inc. 

Rush, A. J., First, M. B., & Blacker, D. (Eds.) 
(2008.) Handbook of Psychiatric Measures, 
Second Edition. Arlington, VA: American 
Psychiatric Publishing, Inc. 

Children develop within microcultures of families 
and neighborhoods that are influenced by larger 
macrocultures characterized by particular 
languages, traditions, social structures, economies, 
values, and attitudes. A concern about assessment 
measures is their generalizability to multiethnic 
cultures. Achenbach & Rescorla (2007) point to 
the need for multicultural research to develop and 
test assessment instruments for use with children 
from multiple cultures. 


Address Cultural Competence 
and EBPs 

You can improve the quahty of your EBP program 
if you ensure that it is culturally competent—that 
is, it adapts to meet the needs of families from 
diverse cultures. It is important, however, to 
ensure that you are informed about culture and 
cultural competence and issues associated with 
these concepts. 


Cultural competence defined 

Cultural competence is an approach to delivering 
services that assumes that services are more 
effective when they are provided within the 
most relevant and meaningful cultural, gender- 
sensitive, and age-appropriate context for the 
people being served. 

The U.S. Surgeon General has defined Cultural 
competence in the most general terms as: 

...the delivery of services responsive to the cultural 
concerns of racial and ethnic minority groups, 
including their languages, histories, traditions, 
beliefs, and values. 

In most cases, cultural competence refers to sets of 
guiding principles developed to increase the ability 
of mental health providers, agencies, or systems to 
meet the needs of diverse communities, including 
racial and ethnic minorities. 

While families, providers, policymakers, and 
administrators have long acknowledged the intrinsic 
value of cultural competence, sufficient research has 
not yet been dedicated to identifying its key 
ingredients. Therefore, the field still struggles to 
define cultural competence, put it into operation, 
and measure it in a manner that is generally 
accepted by researchers and practitioners alike. 
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The word competence is somewhat misleading 
in that it implies that a set of criteria has been 
developed for use in evaluating a program. 

This set of criteria, however, has not yet been 
completely identified; cultural competence is still 
under-researched. In this context, competence 
means that the responsibility to tailor care to 
different cultural groups belongs to the system, 
not to the consumers of services. Every provider 
or administrator who is involved in delivering care 
at every level—from mental health authorities 
to clinical supervisors and practitioners—bears 
responsibility for making their programs accessible, 
appropriate, appeahng, and effective for the 
diverse communities they serve. Many providers 
do this as a matter of course within their practice. 


What culture is and how it affects care 

Broadly defined, a culture is a common heritage 
or set of behefs, norms, and values that a group 
of people shares. People who are placed—either 
by census categories or by identifying themselves— 
into the same racial or ethnic group are often 
assumed to share the same culture. However, this 
can be misleading. 

A great diversity exists within each broad category. 
Individuals may identify with a given racial or 
ethnic culture to varying degrees. Others may 
identify with multiple cultures, including those 
associated with their religion, profession, sexual 
orientation, region, or disability status. 


Culture is dynamic, changing continually, and 
influenced both by people’s beliefs and the 
demands of their environment. Immigrants from 
different parts of the world arrive in the United 
States with their own cultures but gradually begin 
to adapt and develop new, hybrid cultures that 
allow them to function in the dominant culture. 
This process is called acculturation. Even groups 
that have been in the United States for many 
generations may share behefs and practices that 
maintain influences from multiple cultures. This 
complexity necessitates an individualized approach 
to understanding culture and cultural identity in 
the context of mental health services. 

People’s culture influences many aspects of care, 
starting with whether they think care is even 
needed. Culture influences the concerns that 
people bring to the clinical setting, the language 
they use to express those concerns, and the coping 
styles they adopt. 

Culture affects family structure, hving arrangements, 
and the degree of support that people receive 
during difficult times. Culture also influences 
help-seeking behavior, whether people begin with 
a primary care doctor, a mental health program, 
a minister, spiritual advisor, or community elder. 
Einally, culture affects whether people attach 
a stigma to mental healtli problems and how much 
trust they place in providers. 
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The professional culture of agencies, administrators, 
and practitioners influences how care is organized 
and delivered. Cultural influences affect the manner 
in which practitioners ask questions or how they 
interact with families. Culture also affects equally 
important aspects of care that may be less overt, 
such as the following; 

■ The operating hours of an agency; 

■ The importance that staff attaches to reaching 
out to family members and community leaders; 
and 

■ The respect that staff gives to the culture 
of families who enter their doors. 

Knowing how culture influences so many aspects 
of mental health care underscores the importance 
of adapting agency practices to respond to and 
respect the diversity of the surrounding community. 


The need for cultural competence 

For decades, many mental health agencies 
neglected to recognize the growing diversity 
around them. Often, people from nonmajority 
cultures found programs off-putting and hard to 
access. They avoided seeking care, stopped looking 
for care, or—if they managed to find care— 
dropped out. 


Troubling disparities resulted. Many minority 
groups faced lower access to care, lower use of 
care, and poorer quality of care. Disparities are 
most apparent for racial and ethnic minority 
groups, such as: 

■ African Americans; 

■ American Indians and Alaska Natives; 

■ Asian Americans; 

■ Hispanic Americans; and 

■ Native Hawaiians and other Pacific Islanders. 

However, disparities also affect many other groups, 
such as: 

■ Women and men; 

■ Children and older adults; 

■ People from rural and frontier areas; 

■ People with different sexual orientations; and 

■ People with physical or developmental 
disabilities. 

Altogether, those disparities meant that millions 
of people suffered needless disability from 
mental illness. 

Starting in the late 1980s, the mental health 
profession responded to the issue of disparity with 
a new approach to care called cultural competence. 
Originally defined as a set of congruent behaviors, 
attitudes, and policies that come together in a 
system, agency, or among professionals and that 
enables that system, agency, or those professionals 
to work effectively in cross-cultural situations, 
cultural competence was intended to do 
the following: 

■ Improve access to care; 

■ Build trust; and 

■ Promote engagement and retention in care. 
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How cultural competence relates to EBPs 

According to the Surgeon General, evidence-based 
practices are intended for every individual who 
enters care, regardless of his or her culture. But 
many providers ask, “How can we know if EBPs 
apply to a particular ethnic, racial, or cultural 
group if the research supporting those practices 
was done on a very different population?” 

The answer is that we do not yet know how 
these EBPs may apply to various cultural groups. 
However, tlie research base on adapting EBPs 
across multicultural groups is beginning to 
accumulate. In Evidence-Based and Promising 
Practices in this KIT, which describes the EBPs 
in greater detail, formal adaptations of EBPs that 
are being tested for particular ethnic and racial 
groups are described. (See Parent Child 
Interaction Therapy and Parent Management 
Training-Oregon in that booklet.) 


While more research is being conducted, programs 
may try to adjust EBPs to make them accessible 
and effective for cultural groups that differ in 
language or behavior from the original study 
populations used to develop the EBP. 

It is important to be aware, however, that any 
adjustments made to the original EBP model that 
was found to be effective could affect the fidelity 
and outcomes of the EBP. Therefore, it is very 
important to carefully document adjustments 
and to monitor the outcomes that result from 
these adjustments. 
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How to put cultural competence 
into practice 

Since the goal is for all programs is to be more 
culturally competent, we offer a variety of 
straightforward steps to help agency administrators 
respond more effectively to the people it serves. 
These steps apply to all facets of a program; they 
are not restricted to the EBP program. Please note 
that these steps are meant to be illustrative, 
not prescriptive. 

■ Understand the racial, ethnic, and cultural 
demographics of the population served. 

■ Become most familiar with one or two of 
the groups you most commonly encounter. 

■ Create a cultural competence advisory 
committee consisting of youth, family, 
and community organizations. 

■ Translate your forms and brochures. 

■ Offer to match a consumer with a practitioner 
who has a similar background. 

■ Have ready access to trained mental health 
interpreters. 

■ Ask youth and families about their cultural 
backgrounds and identities. 

■ Incorporate cultural awareness into assessment 
and treatment. 


■ Tap into natural networks of support, such as 
the extended family and community groups that 
represent the culture of the youth and family. 

■ Reach out to religious, faith and spiritual 
organizations to encourage referrals or as 
another network of support. 

■ Offer training to staff in culturally responsive 
communication or interviewing skills. 

■ Understand that some behaviors that 
one culture may consider to be signs 
of psychopathology may be acceptable 
in a different culture. 

■ Be aware that people from other cultures 
may hold different beliefs about causes and 
treatments of illness. 

■ Collect and analyze data to examine disparities 
in services. 

■ Designate specific resources for cultural 
competence training. 

■ Include cultural competence in quality- 
assurance and quahty-improvement activities. 

■ Compare outcome data for different cultural 
groups that are receiving EBPs. 

■ Collect and analyze fidelity and outcome 
data for any adjustments made to specific 
components of EBPs to make them more 
sensitive to different cultural groups. 
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Knowledge Informing Transformation 


Guide to EBPs 
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